{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### HyerTS 期待什么格式的数据？\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**摘要：** 在本NoteBook文档中，为了您可以快速掌握HyperTS的使用并顺利获取实验报告，我们将详细地介绍HyperTS在各类时序任务中所需的规范化数据格式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**关键词**：DataFrame; Nested; TimeStamp; Covariables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 预测任务(Forecasting)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在预测任务中，输入数据应该是一个含有时间列(TimeStamp)和变量列的```pandas DataFrame```格式的二维数据表,形式如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "     time_col          var_col_0        var_col_1        var_col_2     ...    var_col_n\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    -\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        -                x                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                -                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                -                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                x                -                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "xxxx-xx-xx xx:xx:xx        x                x                -                    x\n",
    "xxxx-xx-xx xx:xx:xx        -                -                -                    -\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "        -                  -                -                -                    -\n",
    "xxxx-xx-xx xx:xx:xx        x                x                x                    x\n",
    "```\n",
    "\n",
    "**其中，xxxx-xx-xx xx:xx:xx表示时间，(x)表示某个时刻某个变量值，(-)表示缺失值。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意事项**\n",
    "\n",
    "- 在预测任务中的数据中，HyperTS期待也必须含有时间列，列名称不作规范，无论是```ds, ts, timestamp, TimeStamp```还是其他。\n",
    "- 时间列可以具有```pandas.to_datetime```可以识别的任何格式。\n",
    "- 时间可以是无序的，HyperTS可以自动转化为有序数列。\n",
    "- 输入数据的频率满足多种时间粒度，秒(S)、分钟(T)、小时(H)、日(D)、周(W)、月(M)、每年(Y)等等。\n",
    "- 输入数据容忍存在缺失值，缺失点与缺失时间片段, HyperTS将会在数据预处理过程被填充。\n",
    "- 输入数据容忍存在重复行，重复的时间片段, HyperTS将会在被在数据预处理过程被裁剪。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当所解决任务中有辅助建模的数据，我们称之为协变量，它仅需跟附在上述数据的```DataFrame```中，形式如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "     time_col          var_col_0   var_col_1 ... var_col_n     covar_col_0    covar_col_1 ... covar_col_m\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x \n",
    "xxxx-xx-xx xx:xx:xx        x          x              -              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x\n",
    "xxxx-xx-xx xx:xx:xx        -          x              x              x              x               -\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        -          -              -              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "        -                  -          -              -              -              -               -\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "```\n",
    "**其中，covar_col_i (i=1,2,..,m)表示协变量。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意事项**\n",
    "\n",
    "- 协变量可以是数值型(连续变量)也可以是类别型(离散变量)。\n",
    "- 协变量也容忍存在缺失值和重复值，不需要自己处理。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**举个例子**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>val_0</th>\n",
       "      <th>val_1</th>\n",
       "      <th>val_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-02-01 00:00:00</td>\n",
       "      <td>-0.082988</td>\n",
       "      <td>0.5</td>\n",
       "      <td>-2.013005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-02-01 01:00:00</td>\n",
       "      <td>-0.413941</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.684124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022-02-01 02:00:00</td>\n",
       "      <td>-0.503602</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.261157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022-02-01 03:00:00</td>\n",
       "      <td>-0.955423</td>\n",
       "      <td>0.9</td>\n",
       "      <td>-0.125037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022-02-01 04:00:00</td>\n",
       "      <td>0.247220</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.148305</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            timestamp     val_0  val_1     val_2\n",
       "0 2022-02-01 00:00:00 -0.082988    0.5 -2.013005\n",
       "1 2022-02-01 01:00:00 -0.413941    0.2  0.684124\n",
       "2 2022-02-01 02:00:00 -0.503602    NaN  0.261157\n",
       "3 2022-02-01 03:00:00 -0.955423    0.9 -0.125037\n",
       "4 2022-02-01 04:00:00  0.247220    0.0  0.148305"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "size=5\n",
    "\n",
    "#没有协变量\n",
    "df_no_covariate = pd.DataFrame({\n",
    "    'timestamp': pd.date_range(start='2022-02-01', periods=5, freq='H'),\n",
    "    'val_0': np.random.normal(size=size),\n",
    "    'val_1': [0.5, 0.2, np.nan, 0.9, 0.0],\n",
    "    'val_2': np.random.normal(size=size),\n",
    "})\n",
    "\n",
    "df_no_covariate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>val_0</th>\n",
       "      <th>val_1</th>\n",
       "      <th>val_2</th>\n",
       "      <th>covar_0</th>\n",
       "      <th>covar_1</th>\n",
       "      <th>covar_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-02-01</td>\n",
       "      <td>-1.145418</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>a</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-02-02</td>\n",
       "      <td>-1.466120</td>\n",
       "      <td>52.0</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.4</td>\n",
       "      <td>a</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022-02-03</td>\n",
       "      <td>0.116391</td>\n",
       "      <td>34.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.2</td>\n",
       "      <td>b</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022-02-04</td>\n",
       "      <td>-0.608921</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.7</td>\n",
       "      <td>b</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022-02-05</td>\n",
       "      <td>-1.250787</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>b</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   timestamp     val_0  val_1  val_2  covar_0 covar_1  covar_2\n",
       "0 2022-02-01 -1.145418   12.0    0.5      0.2       a      1.0\n",
       "1 2022-02-02 -1.466120   52.0    0.2      0.4       a      2.0\n",
       "2 2022-02-03  0.116391   34.0    NaN      0.2       b      2.0\n",
       "3 2022-02-04 -0.608921    NaN    0.9      0.7       b      NaN\n",
       "4 2022-02-05 -1.250787  100.0    0.0      0.1       b      3.0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 有协变量\n",
    "df_with_covariate = pd.DataFrame({\n",
    "    'timestamp': pd.date_range(start='2022-02-01', periods=size, freq='D'),\n",
    "    'val_0': np.random.normal(size=size),\n",
    "    'val_1': [12, 52, 34, np.nan, 100],\n",
    "    'val_2': [0.5, 0.2, np.nan, 0.9, 0.0],\n",
    "    'covar_0': [0.2, 0.4, 0.2, 0.7, 0.1],\n",
    "    'covar_1': ['a', 'a', 'b', 'b', 'b'],\n",
    "    'covar_2': [1, 2, 2, None, 3], \n",
    "})\n",
    "\n",
    "df_with_covariate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**加载HyperTS的数据集**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hyperts.datasets import (load_random_univariate_forecast_dataset, \n",
    "                              load_random_multivariate_forecast_dataset,\n",
    "                              load_network_traffic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>id</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013-01-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.922501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013-01-02</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.496957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013-01-03</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.252253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013-01-04</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.103628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013-01-05</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.853269</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ds   id     value\n",
       "0 2013-01-01  0.0  0.922501\n",
       "1 2013-01-02  0.0  0.496957\n",
       "2 2013-01-03  1.0  0.252253\n",
       "3 2013-01-04  0.0  0.103628\n",
       "4 2013-01-05  0.0  0.853269"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df0 = load_random_univariate_forecast_dataset(return_X_y=False)\n",
    "df0.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个数据集中，```ds```是时间列名称，时间频率是天(D), 目标变量(预测变量)是```value```列，```id```列是协变量列，是一个单变量时间序列任务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>Var_1</th>\n",
       "      <th>Var_2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-02-10 14:32:05.430194</td>\n",
       "      <td>0.461891</td>\n",
       "      <td>0.866003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2022-02-11 14:32:05.430194</td>\n",
       "      <td>1.761956</td>\n",
       "      <td>2.608567</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2022-02-12 14:32:05.430194</td>\n",
       "      <td>2.591447</td>\n",
       "      <td>2.807734</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2022-02-13 14:32:05.430194</td>\n",
       "      <td>3.622879</td>\n",
       "      <td>4.596779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2022-02-14 14:32:05.430194</td>\n",
       "      <td>4.098946</td>\n",
       "      <td>4.340272</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          ds     Var_1     Var_2\n",
       "0 2022-02-10 14:32:05.430194  0.461891  0.866003\n",
       "1 2022-02-11 14:32:05.430194  1.761956  2.608567\n",
       "2 2022-02-12 14:32:05.430194  2.591447  2.807734\n",
       "3 2022-02-13 14:32:05.430194  3.622879  4.596779\n",
       "4 2022-02-14 14:32:05.430194  4.098946  4.340272"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = load_random_multivariate_forecast_dataset(return_X_y=False)\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个数据集中，```ds```是时间列名称，时间频率是天(D), 目标变量(预测变量)是```Val_1, Val_2```两列，没有协变量，是一个多变量时间序列任务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TimeStamp</th>\n",
       "      <th>Var_1</th>\n",
       "      <th>Var_2</th>\n",
       "      <th>Var_3</th>\n",
       "      <th>Var_4</th>\n",
       "      <th>Var_5</th>\n",
       "      <th>Var_6</th>\n",
       "      <th>HourSin</th>\n",
       "      <th>WeekCos</th>\n",
       "      <th>CBWD</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2021-03-01 00:00:00</td>\n",
       "      <td>0.7534</td>\n",
       "      <td>3.375</td>\n",
       "      <td>10.195</td>\n",
       "      <td>1.4490</td>\n",
       "      <td>19174.977</td>\n",
       "      <td>286443.880</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021-03-01 01:00:00</td>\n",
       "      <td>0.3376</td>\n",
       "      <td>2.414</td>\n",
       "      <td>3.920</td>\n",
       "      <td>0.4065</td>\n",
       "      <td>7529.263</td>\n",
       "      <td>178930.450</td>\n",
       "      <td>0.258819</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2021-03-01 02:00:00</td>\n",
       "      <td>0.2032</td>\n",
       "      <td>1.654</td>\n",
       "      <td>3.318</td>\n",
       "      <td>0.2142</td>\n",
       "      <td>3310.539</td>\n",
       "      <td>42296.164</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2021-03-01 03:00:00</td>\n",
       "      <td>0.2420</td>\n",
       "      <td>1.393</td>\n",
       "      <td>3.148</td>\n",
       "      <td>0.2312</td>\n",
       "      <td>4535.464</td>\n",
       "      <td>26220.232</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2021-03-01 04:00:00</td>\n",
       "      <td>0.1940</td>\n",
       "      <td>1.429</td>\n",
       "      <td>3.215</td>\n",
       "      <td>0.2157</td>\n",
       "      <td>2732.911</td>\n",
       "      <td>27990.348</td>\n",
       "      <td>0.866025</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NW</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             TimeStamp   Var_1  Var_2   Var_3   Var_4      Var_5       Var_6  \\\n",
       "0  2021-03-01 00:00:00  0.7534  3.375  10.195  1.4490  19174.977  286443.880   \n",
       "1  2021-03-01 01:00:00  0.3376  2.414   3.920  0.4065   7529.263  178930.450   \n",
       "2  2021-03-01 02:00:00  0.2032  1.654   3.318  0.2142   3310.539   42296.164   \n",
       "3  2021-03-01 03:00:00  0.2420  1.393   3.148  0.2312   4535.464   26220.232   \n",
       "4  2021-03-01 04:00:00  0.1940  1.429   3.215  0.2157   2732.911   27990.348   \n",
       "\n",
       "    HourSin  WeekCos CBWD  \n",
       "0  0.000000      1.0   NW  \n",
       "1  0.258819      1.0   NW  \n",
       "2  0.500000      1.0   NW  \n",
       "3  0.707107      1.0   NW  \n",
       "4  0.866025      1.0   NW  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = load_network_traffic(return_X_y=False)\n",
    "df2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个数据集中，```TimeStamp```是时间列名称，时间频率是小时(H), 目标变量(预测变量)是```Val_1, Val_2，Val_3, Val_4，Val_5, Val_6```六列，协变量是HourSin, WeekCos, CBWD, 其中HourSin, WeekCos是数值型变量，CBWD是类别型变量，是一个多变量时间序列任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<bar>\n",
    "<bar>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 分类任务(Classification/Regression)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "具体表现在输入数据的形式为含有目标列(target)及特征列的嵌套(nested) ```pandas DataFrame```格式的二维数据表，形式如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    " var_col_0       var_col_1   ...    var_col_n     target\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      0\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      0\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      1\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      1\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      1\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      2\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      2\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      2\n",
    "x,x,x,...,x     x,x,x,...,x        x,x,x,...,x      2\n",
    "```\n",
    "\n",
    "**其中，x,x,x,...,x表示某样本在len(x,x,x,...,x)长度的时间片段某变量随时间的波动情况。(x)表示某个时刻某个变量值。**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意事项**\n",
    "\n",
    "- 分类或者回归任务是针对一个样本判断其行为，故与预测任务的数据形式不同，预测数据每一行表示一个时间点各个变量的值，而分类或回归数据每一行表示一个样本，而每一个cell, 即```x,x,x,...,x```表示某样本在len(```x,x,x,...,x```)长度的时间片段某变量随时间波动的情况。每个样本根据各个变量的序列行为判别```target```。\n",
    "- 直觉上，```pandas DadaFrame```是一二维数据表，每一个cell储存一个数值，现在我们储存一个序列，从而将三维数据嵌套在二维数据表中，这也是我们称之为 ```nested DataFrame```的原因。\n",
    "- 分类或回归任务的目标是判别每一个样本的类别或者行为，故数据的走势是关键特质，所以为了简单起见，我们在存储时省略去TimeStamp的信息。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**举个例子**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>var_0</th>\n",
       "      <th>var_1</th>\n",
       "      <th>var_2</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0    1.517646\n",
       "1   -0.130092\n",
       "2    0.041960\n",
       "3   ...</td>\n",
       "      <td>0    1.339915\n",
       "1    0.905619\n",
       "2    0.661068\n",
       "3   ...</td>\n",
       "      <td>0    0.458820\n",
       "1   -0.205814\n",
       "2   -0.779682\n",
       "3   ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0   -2.202468\n",
       "1    0.586854\n",
       "2    0.741383\n",
       "3   ...</td>\n",
       "      <td>0    1.792085\n",
       "1    0.169211\n",
       "2   -1.507392\n",
       "3   ...</td>\n",
       "      <td>0   -0.191859\n",
       "1    1.491976\n",
       "2    1.643508\n",
       "3   ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0   -0.257921\n",
       "1    0.277949\n",
       "2    0.113696\n",
       "3   ...</td>\n",
       "      <td>0    0.862955\n",
       "1   -2.383729\n",
       "2    0.685731\n",
       "3   ...</td>\n",
       "      <td>0   -0.481607\n",
       "1   -3.252144\n",
       "2    0.705657\n",
       "3   ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0    0.474522\n",
       "1    0.812421\n",
       "2    0.945904\n",
       "3   ...</td>\n",
       "      <td>0   -0.689202\n",
       "1   -1.111300\n",
       "2   -0.905664\n",
       "3   ...</td>\n",
       "      <td>0   -0.922138\n",
       "1    0.288886\n",
       "2   -1.289592\n",
       "3   ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0   -0.838331\n",
       "1   -0.445160\n",
       "2   -0.417691\n",
       "3   ...</td>\n",
       "      <td>0    1.117217\n",
       "1   -0.482835\n",
       "2    1.816897\n",
       "3   ...</td>\n",
       "      <td>0    0.255738\n",
       "1   -0.702608\n",
       "2    3.110447\n",
       "3   ...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0    0.342809\n",
       "1   -0.018744\n",
       "2   -1.681167\n",
       "3   ...</td>\n",
       "      <td>0    0.527534\n",
       "1    1.360901\n",
       "2    0.865364\n",
       "3   ...</td>\n",
       "      <td>0    0.978863\n",
       "1    0.581206\n",
       "2   -1.267197\n",
       "3   ...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               var_0  \\\n",
       "0  0    1.517646\n",
       "1   -0.130092\n",
       "2    0.041960\n",
       "3   ...   \n",
       "1  0   -2.202468\n",
       "1    0.586854\n",
       "2    0.741383\n",
       "3   ...   \n",
       "2  0   -0.257921\n",
       "1    0.277949\n",
       "2    0.113696\n",
       "3   ...   \n",
       "3  0    0.474522\n",
       "1    0.812421\n",
       "2    0.945904\n",
       "3   ...   \n",
       "4  0   -0.838331\n",
       "1   -0.445160\n",
       "2   -0.417691\n",
       "3   ...   \n",
       "5  0    0.342809\n",
       "1   -0.018744\n",
       "2   -1.681167\n",
       "3   ...   \n",
       "\n",
       "                                               var_1  \\\n",
       "0  0    1.339915\n",
       "1    0.905619\n",
       "2    0.661068\n",
       "3   ...   \n",
       "1  0    1.792085\n",
       "1    0.169211\n",
       "2   -1.507392\n",
       "3   ...   \n",
       "2  0    0.862955\n",
       "1   -2.383729\n",
       "2    0.685731\n",
       "3   ...   \n",
       "3  0   -0.689202\n",
       "1   -1.111300\n",
       "2   -0.905664\n",
       "3   ...   \n",
       "4  0    1.117217\n",
       "1   -0.482835\n",
       "2    1.816897\n",
       "3   ...   \n",
       "5  0    0.527534\n",
       "1    1.360901\n",
       "2    0.865364\n",
       "3   ...   \n",
       "\n",
       "                                               var_2  y  \n",
       "0  0    0.458820\n",
       "1   -0.205814\n",
       "2   -0.779682\n",
       "3   ...  0  \n",
       "1  0   -0.191859\n",
       "1    1.491976\n",
       "2    1.643508\n",
       "3   ...  0  \n",
       "2  0   -0.481607\n",
       "1   -3.252144\n",
       "2    0.705657\n",
       "3   ...  1  \n",
       "3  0   -0.922138\n",
       "1    0.288886\n",
       "2   -1.289592\n",
       "3   ...  1  \n",
       "4  0    0.255738\n",
       "1   -0.702608\n",
       "2    3.110447\n",
       "3   ...  2  \n",
       "5  0    0.978863\n",
       "1    0.581206\n",
       "2   -1.267197\n",
       "3   ...  2  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "size=10\n",
    "\n",
    "df = pd.DataFrame({\n",
    "    'var_0': [pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size))],\n",
    "    'var_1': [pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size))],\n",
    "    'var_2': [pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size)),\n",
    "              pd.Series(np.random.normal(size=size)), pd.Series(np.random.normal(size=size))],\n",
    "    'y': [0, 0, 1, 1, 2, 2], \n",
    "})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**加载HyperTS的数据集**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hyperts.datasets import load_arrow_head, load_basic_motions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Var_1</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0     -1.9630\n",
       "1     -1.9578\n",
       "2     -1.9561\n",
       "3   ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0     -1.7746\n",
       "1     -1.7740\n",
       "2     -1.7766\n",
       "3   ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0     -1.8660\n",
       "1     -1.8420\n",
       "2     -1.8350\n",
       "3   ...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0     -2.0738\n",
       "1     -2.0733\n",
       "2     -2.0446\n",
       "3   ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0     -1.7463\n",
       "1     -1.7413\n",
       "2     -1.7227\n",
       "3   ...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Var_1 target\n",
       "0  0     -1.9630\n",
       "1     -1.9578\n",
       "2     -1.9561\n",
       "3   ...      0\n",
       "1  0     -1.7746\n",
       "1     -1.7740\n",
       "2     -1.7766\n",
       "3   ...      1\n",
       "2  0     -1.8660\n",
       "1     -1.8420\n",
       "2     -1.8350\n",
       "3   ...      2\n",
       "3  0     -2.0738\n",
       "1     -2.0733\n",
       "2     -2.0446\n",
       "3   ...      0\n",
       "4  0     -1.7463\n",
       "1     -1.7413\n",
       "2     -1.7227\n",
       "3   ...      1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df0 = load_arrow_head(return_X_y=False)\n",
    "df0.head()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['0', '1', '2'], dtype=object)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df0.target.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个数据集中，```target```是目标列, 包含三个类别['0', '1', '2']， ```Var_1```是特征变量，有且仅有一个，故是一个单变量多分类任务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Var_1</th>\n",
       "      <th>Var_2</th>\n",
       "      <th>Var_3</th>\n",
       "      <th>Var_4</th>\n",
       "      <th>Var_5</th>\n",
       "      <th>Var_6</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0     0.079106\n",
       "1     0.079106\n",
       "2    -0.903497\n",
       "3...</td>\n",
       "      <td>0     0.394032\n",
       "1     0.394032\n",
       "2    -3.666397\n",
       "3...</td>\n",
       "      <td>0     0.551444\n",
       "1     0.551444\n",
       "2    -0.282844\n",
       "3...</td>\n",
       "      <td>0     0.351565\n",
       "1     0.351565\n",
       "2    -0.095881\n",
       "3...</td>\n",
       "      <td>0     0.023970\n",
       "1     0.023970\n",
       "2    -0.319605\n",
       "3...</td>\n",
       "      <td>0     0.633883\n",
       "1     0.633883\n",
       "2     0.972131\n",
       "3...</td>\n",
       "      <td>standing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0     0.377751\n",
       "1     0.377751\n",
       "2     2.952965\n",
       "3...</td>\n",
       "      <td>0    -0.610850\n",
       "1    -0.610850\n",
       "2     0.970717\n",
       "3...</td>\n",
       "      <td>0    -0.147376\n",
       "1    -0.147376\n",
       "2    -5.962515\n",
       "3...</td>\n",
       "      <td>0    -0.103872\n",
       "1    -0.103872\n",
       "2    -7.593275\n",
       "3...</td>\n",
       "      <td>0    -0.109198\n",
       "1    -0.109198\n",
       "2    -0.697804\n",
       "3...</td>\n",
       "      <td>0    -0.037287\n",
       "1    -0.037287\n",
       "2    -2.865789\n",
       "3...</td>\n",
       "      <td>standing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0    -0.813905\n",
       "1    -0.813905\n",
       "2    -0.424628\n",
       "3...</td>\n",
       "      <td>0     0.825666\n",
       "1     0.825666\n",
       "2    -1.305033\n",
       "3...</td>\n",
       "      <td>0     0.032712\n",
       "1     0.032712\n",
       "2     0.826170\n",
       "3...</td>\n",
       "      <td>0     0.021307\n",
       "1     0.021307\n",
       "2    -0.372872\n",
       "3...</td>\n",
       "      <td>0     0.122515\n",
       "1     0.122515\n",
       "2    -0.045277\n",
       "3...</td>\n",
       "      <td>0     0.775041\n",
       "1     0.775041\n",
       "2     0.383526\n",
       "3...</td>\n",
       "      <td>standing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0     0.289855\n",
       "1     0.289855\n",
       "2    -0.669185\n",
       "3...</td>\n",
       "      <td>0     0.284130\n",
       "1     0.284130\n",
       "2    -0.210466\n",
       "3...</td>\n",
       "      <td>0     0.213680\n",
       "1     0.213680\n",
       "2     0.252267\n",
       "3...</td>\n",
       "      <td>0    -0.314278\n",
       "1    -0.314278\n",
       "2     0.018644\n",
       "3...</td>\n",
       "      <td>0     0.074574\n",
       "1     0.074574\n",
       "2     0.007990\n",
       "3...</td>\n",
       "      <td>0    -0.079901\n",
       "1    -0.079901\n",
       "2     0.237040\n",
       "3...</td>\n",
       "      <td>standing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0    -0.123238\n",
       "1    -0.123238\n",
       "2    -0.249547\n",
       "3...</td>\n",
       "      <td>0     0.379341\n",
       "1     0.379341\n",
       "2     0.541501\n",
       "3...</td>\n",
       "      <td>0    -0.286006\n",
       "1    -0.286006\n",
       "2     0.208420\n",
       "3...</td>\n",
       "      <td>0    -0.098545\n",
       "1    -0.098545\n",
       "2    -0.023970\n",
       "3...</td>\n",
       "      <td>0     0.058594\n",
       "1     0.058594\n",
       "2     0.175783\n",
       "3...</td>\n",
       "      <td>0    -0.074574\n",
       "1    -0.074574\n",
       "2     0.114525\n",
       "3...</td>\n",
       "      <td>standing</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Var_1  \\\n",
       "0  0     0.079106\n",
       "1     0.079106\n",
       "2    -0.903497\n",
       "3...   \n",
       "1  0     0.377751\n",
       "1     0.377751\n",
       "2     2.952965\n",
       "3...   \n",
       "2  0    -0.813905\n",
       "1    -0.813905\n",
       "2    -0.424628\n",
       "3...   \n",
       "3  0     0.289855\n",
       "1     0.289855\n",
       "2    -0.669185\n",
       "3...   \n",
       "4  0    -0.123238\n",
       "1    -0.123238\n",
       "2    -0.249547\n",
       "3...   \n",
       "\n",
       "                                               Var_2  \\\n",
       "0  0     0.394032\n",
       "1     0.394032\n",
       "2    -3.666397\n",
       "3...   \n",
       "1  0    -0.610850\n",
       "1    -0.610850\n",
       "2     0.970717\n",
       "3...   \n",
       "2  0     0.825666\n",
       "1     0.825666\n",
       "2    -1.305033\n",
       "3...   \n",
       "3  0     0.284130\n",
       "1     0.284130\n",
       "2    -0.210466\n",
       "3...   \n",
       "4  0     0.379341\n",
       "1     0.379341\n",
       "2     0.541501\n",
       "3...   \n",
       "\n",
       "                                               Var_3  \\\n",
       "0  0     0.551444\n",
       "1     0.551444\n",
       "2    -0.282844\n",
       "3...   \n",
       "1  0    -0.147376\n",
       "1    -0.147376\n",
       "2    -5.962515\n",
       "3...   \n",
       "2  0     0.032712\n",
       "1     0.032712\n",
       "2     0.826170\n",
       "3...   \n",
       "3  0     0.213680\n",
       "1     0.213680\n",
       "2     0.252267\n",
       "3...   \n",
       "4  0    -0.286006\n",
       "1    -0.286006\n",
       "2     0.208420\n",
       "3...   \n",
       "\n",
       "                                               Var_4  \\\n",
       "0  0     0.351565\n",
       "1     0.351565\n",
       "2    -0.095881\n",
       "3...   \n",
       "1  0    -0.103872\n",
       "1    -0.103872\n",
       "2    -7.593275\n",
       "3...   \n",
       "2  0     0.021307\n",
       "1     0.021307\n",
       "2    -0.372872\n",
       "3...   \n",
       "3  0    -0.314278\n",
       "1    -0.314278\n",
       "2     0.018644\n",
       "3...   \n",
       "4  0    -0.098545\n",
       "1    -0.098545\n",
       "2    -0.023970\n",
       "3...   \n",
       "\n",
       "                                               Var_5  \\\n",
       "0  0     0.023970\n",
       "1     0.023970\n",
       "2    -0.319605\n",
       "3...   \n",
       "1  0    -0.109198\n",
       "1    -0.109198\n",
       "2    -0.697804\n",
       "3...   \n",
       "2  0     0.122515\n",
       "1     0.122515\n",
       "2    -0.045277\n",
       "3...   \n",
       "3  0     0.074574\n",
       "1     0.074574\n",
       "2     0.007990\n",
       "3...   \n",
       "4  0     0.058594\n",
       "1     0.058594\n",
       "2     0.175783\n",
       "3...   \n",
       "\n",
       "                                               Var_6    target  \n",
       "0  0     0.633883\n",
       "1     0.633883\n",
       "2     0.972131\n",
       "3...  standing  \n",
       "1  0    -0.037287\n",
       "1    -0.037287\n",
       "2    -2.865789\n",
       "3...  standing  \n",
       "2  0     0.775041\n",
       "1     0.775041\n",
       "2     0.383526\n",
       "3...  standing  \n",
       "3  0    -0.079901\n",
       "1    -0.079901\n",
       "2     0.237040\n",
       "3...  standing  \n",
       "4  0    -0.074574\n",
       "1    -0.074574\n",
       "2     0.114525\n",
       "3...  standing  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = load_basic_motions(return_X_y=False)\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['standing', 'running', 'walking', 'badminton'], dtype=object)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1.target.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个数据集中, ```target```是目标列，包含四个类别['standing', 'running', 'walking', 'badminton'], ```Var_1, Var_2, Var_3, Var_4, Var_5, Var_6```是特征变量，共六个，故是一个多变量多分类任务。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**提示**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当拿到的原始数据是```numpy.array```形式时，我们如何将其转化为嵌套的```pandas.DataFrame```数据呢？例如如下数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "nb_samples = 100\n",
    "series_length = 72\n",
    "nb_variables = 6\n",
    "nb_classes = 4\n",
    "\n",
    "X = np.random.normal(size=nb_samples*series_length*nb_variables).reshape(nb_samples, series_length, nb_variables)\n",
    "y = np.random.randint(low=0, high=nb_classes, size=nb_samples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((100, 72, 6), (100,), array([0, 1, 2, 3]))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, y.shape, np.unique(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由数据可知，该数据包含了```100```个样本，每个样本有```6```个变量，而每个变量是长度为```72```的时间序列。y共有```4```个类别。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "面对这样的情况，HyperTS提供了相关变换的工具函数```from_3d_array_to_nested_df```："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from hyperts.toolbox import from_3d_array_to_nested_df\n",
    "\n",
    "df_X = from_3d_array_to_nested_df(data=X)\n",
    "df_y = pd.DataFrame({'y': y})\n",
    "df = pd.concat([df_X, df_y], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Var_0</th>\n",
       "      <th>Var_1</th>\n",
       "      <th>Var_2</th>\n",
       "      <th>Var_3</th>\n",
       "      <th>Var_4</th>\n",
       "      <th>Var_5</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0    -0.860411\n",
       "1     1.862670\n",
       "2    -0.921505\n",
       "3...</td>\n",
       "      <td>0     1.398394\n",
       "1     0.660091\n",
       "2     0.128765\n",
       "3...</td>\n",
       "      <td>0     0.525402\n",
       "1    -0.308939\n",
       "2    -0.404762\n",
       "3...</td>\n",
       "      <td>0     0.300226\n",
       "1     0.111569\n",
       "2    -0.659606\n",
       "3...</td>\n",
       "      <td>0     0.245592\n",
       "1     1.196472\n",
       "2     0.037356\n",
       "3...</td>\n",
       "      <td>0    -0.192188\n",
       "1    -0.894914\n",
       "2     0.590388\n",
       "3...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0     0.715266\n",
       "1     0.315416\n",
       "2     0.154454\n",
       "3...</td>\n",
       "      <td>0     0.176125\n",
       "1    -1.172978\n",
       "2     0.456093\n",
       "3...</td>\n",
       "      <td>0     1.041922\n",
       "1    -0.385470\n",
       "2     0.916923\n",
       "3...</td>\n",
       "      <td>0     0.618068\n",
       "1    -0.830899\n",
       "2    -2.476492\n",
       "3...</td>\n",
       "      <td>0     1.242938\n",
       "1    -0.375228\n",
       "2    -1.948744\n",
       "3...</td>\n",
       "      <td>0     0.309130\n",
       "1     0.872872\n",
       "2    -0.512654\n",
       "3...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0    -0.190163\n",
       "1    -0.636367\n",
       "2     0.557982\n",
       "3...</td>\n",
       "      <td>0    -0.304956\n",
       "1     0.346306\n",
       "2    -0.312518\n",
       "3...</td>\n",
       "      <td>0     0.031167\n",
       "1     0.154234\n",
       "2    -0.368498\n",
       "3...</td>\n",
       "      <td>0     0.038083\n",
       "1    -0.346858\n",
       "2    -0.121023\n",
       "3...</td>\n",
       "      <td>0     0.977429\n",
       "1     0.084071\n",
       "2     0.226736\n",
       "3...</td>\n",
       "      <td>0    -0.225884\n",
       "1    -0.653812\n",
       "2     0.392387\n",
       "3...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0    -0.805321\n",
       "1    -0.985039\n",
       "2     0.747402\n",
       "3...</td>\n",
       "      <td>0    -0.750069\n",
       "1    -0.495475\n",
       "2    -0.335203\n",
       "3...</td>\n",
       "      <td>0     1.020891\n",
       "1    -1.145493\n",
       "2     0.289771\n",
       "3...</td>\n",
       "      <td>0     1.354475\n",
       "1    -1.320292\n",
       "2    -1.861200\n",
       "3...</td>\n",
       "      <td>0    -0.189189\n",
       "1     1.339734\n",
       "2    -0.354420\n",
       "3...</td>\n",
       "      <td>0     0.762758\n",
       "1     0.559134\n",
       "2    -1.123743\n",
       "3...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0    -0.358629\n",
       "1    -1.338477\n",
       "2     0.547015\n",
       "3...</td>\n",
       "      <td>0    -1.078313\n",
       "1    -0.145033\n",
       "2     0.044039\n",
       "3...</td>\n",
       "      <td>0     0.122722\n",
       "1    -0.038923\n",
       "2    -1.711621\n",
       "3...</td>\n",
       "      <td>0     0.212634\n",
       "1     0.379226\n",
       "2    -1.555363\n",
       "3...</td>\n",
       "      <td>0    -0.348793\n",
       "1    -1.117400\n",
       "2    -1.463423\n",
       "3...</td>\n",
       "      <td>0     2.460091\n",
       "1     0.306163\n",
       "2    -0.568233\n",
       "3...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Var_0  \\\n",
       "0  0    -0.860411\n",
       "1     1.862670\n",
       "2    -0.921505\n",
       "3...   \n",
       "1  0     0.715266\n",
       "1     0.315416\n",
       "2     0.154454\n",
       "3...   \n",
       "2  0    -0.190163\n",
       "1    -0.636367\n",
       "2     0.557982\n",
       "3...   \n",
       "3  0    -0.805321\n",
       "1    -0.985039\n",
       "2     0.747402\n",
       "3...   \n",
       "4  0    -0.358629\n",
       "1    -1.338477\n",
       "2     0.547015\n",
       "3...   \n",
       "\n",
       "                                               Var_1  \\\n",
       "0  0     1.398394\n",
       "1     0.660091\n",
       "2     0.128765\n",
       "3...   \n",
       "1  0     0.176125\n",
       "1    -1.172978\n",
       "2     0.456093\n",
       "3...   \n",
       "2  0    -0.304956\n",
       "1     0.346306\n",
       "2    -0.312518\n",
       "3...   \n",
       "3  0    -0.750069\n",
       "1    -0.495475\n",
       "2    -0.335203\n",
       "3...   \n",
       "4  0    -1.078313\n",
       "1    -0.145033\n",
       "2     0.044039\n",
       "3...   \n",
       "\n",
       "                                               Var_2  \\\n",
       "0  0     0.525402\n",
       "1    -0.308939\n",
       "2    -0.404762\n",
       "3...   \n",
       "1  0     1.041922\n",
       "1    -0.385470\n",
       "2     0.916923\n",
       "3...   \n",
       "2  0     0.031167\n",
       "1     0.154234\n",
       "2    -0.368498\n",
       "3...   \n",
       "3  0     1.020891\n",
       "1    -1.145493\n",
       "2     0.289771\n",
       "3...   \n",
       "4  0     0.122722\n",
       "1    -0.038923\n",
       "2    -1.711621\n",
       "3...   \n",
       "\n",
       "                                               Var_3  \\\n",
       "0  0     0.300226\n",
       "1     0.111569\n",
       "2    -0.659606\n",
       "3...   \n",
       "1  0     0.618068\n",
       "1    -0.830899\n",
       "2    -2.476492\n",
       "3...   \n",
       "2  0     0.038083\n",
       "1    -0.346858\n",
       "2    -0.121023\n",
       "3...   \n",
       "3  0     1.354475\n",
       "1    -1.320292\n",
       "2    -1.861200\n",
       "3...   \n",
       "4  0     0.212634\n",
       "1     0.379226\n",
       "2    -1.555363\n",
       "3...   \n",
       "\n",
       "                                               Var_4  \\\n",
       "0  0     0.245592\n",
       "1     1.196472\n",
       "2     0.037356\n",
       "3...   \n",
       "1  0     1.242938\n",
       "1    -0.375228\n",
       "2    -1.948744\n",
       "3...   \n",
       "2  0     0.977429\n",
       "1     0.084071\n",
       "2     0.226736\n",
       "3...   \n",
       "3  0    -0.189189\n",
       "1     1.339734\n",
       "2    -0.354420\n",
       "3...   \n",
       "4  0    -0.348793\n",
       "1    -1.117400\n",
       "2    -1.463423\n",
       "3...   \n",
       "\n",
       "                                               Var_5  y  \n",
       "0  0    -0.192188\n",
       "1    -0.894914\n",
       "2     0.590388\n",
       "3...  3  \n",
       "1  0     0.309130\n",
       "1     0.872872\n",
       "2    -0.512654\n",
       "3...  0  \n",
       "2  0    -0.225884\n",
       "1    -0.653812\n",
       "2     0.392387\n",
       "3...  2  \n",
       "3  0     0.762758\n",
       "1     0.559134\n",
       "2    -1.123743\n",
       "3...  2  \n",
       "4  0     2.460091\n",
       "1     0.306163\n",
       "2    -0.568233\n",
       "3...  3  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 异常检测任务(Anomaly Detection)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与预测任务相似，输入数据应该是一个含有时间列(TimeStamp)和变量列的```pandas DataFrame```格式的二维数据表, 其应该包含时间戳列(``time_col``)，一个或多个变量列(``var_col_0``, ``var_col_1``, ``var_col_2``,... ``var_col_n``)，如果有协变量，也可包含一个或多个协变量(``covar_col_0``, ``covar_col_1``, ``covar_col_2``,... ``covar_col_m``)，形式如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "     time_col          var_col_0   var_col_1 ... var_col_n     covar_col_0    covar_col_1 ... covar_col_m\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x \n",
    "xxxx-xx-xx xx:xx:xx        x          x              -              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x\n",
    "xxxx-xx-xx xx:xx:xx        -          x              x              x              x               -\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        -          -              -              x              x               x\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "        -                  -          -              -              -              -               -\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，以上数据也可以包含真实标签，这将有助于模型选择和超参数搜索过程。形式如下所示:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "     time_col          var_col_0   var_col_1 ... var_col_n     covar_col_0    covar_col_1 ... covar_col_m       anomaly       \n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               1 \n",
    "xxxx-xx-xx xx:xx:xx        x          x              -              x              x               x               0\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x               0\n",
    "xxxx-xx-xx xx:xx:xx        -          x              x              x              x               -               0\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               1\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x               0\n",
    "xxxx-xx-xx xx:xx:xx        x          -              x              x              x               x               0\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              -               x               1\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               0\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               0\n",
    "xxxx-xx-xx xx:xx:xx        -          -              -              x              x               x               1\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               0\n",
    "        -                  -          -              -              -              -               -\n",
    "xxxx-xx-xx xx:xx:xx        x          x              x              x              x               x               0\n",
    "```\n",
    "\n",
    "\n",
    "where ``anomaly`` is anomaly label column."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注意事项**\n",
    "\n",
    "   当数据包含真实标签时，优化评估采用真实标签。否则，生成的伪标签将被采用。"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "29dd11bf16d40fe19950dcc2f06dd773fb6bc4491ac296fb211bfed7a4a532da"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}